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INTRODUCTION 


This  paper  is  about  the  Operational  Decision  Aids  Project 
of  the  Office  of  Naval  Research.  It  is  intended  to  provide  an  over- 
view of  ODA:  its  objectives,  history,  approach,  and  progress;  and  its 

products  and  how  they  relate  to  one  another.  (A  separate  memorandum 
containing  a critical  review  of  contractors'  progress  as  of  mid-December 
1976  has  already  had  a limited  distribution.) 


BACKGROUND 

ODA  came  into  being  in  mid-1973  as  the  result  of  an  initiative  by 
four  ONR  programs.  The  objectives  of  ODA  were  to  bring  to  bear  several 
disparate  but  related  technologies  on  the  development  of  Navy  command 
and  control  systems.  More  specifically,  ODA  promised  to  address  issues 
having  to  do  with  decisions  made  by  relatively  senior  officers  and  their 
staffs,  e.g.,  task  force  commanders.  Program  managers  at  ONR  were  aware 
of  a long  and  not  always  satisfactory  history  of  RSD  aimed  at  automating 
certain  elements  of  naval  command  and  control  systems.  What  was  different 
about  the  present  effort  was  its  emphasis  on  recent  technical  developments 
in  four  areas — computer  science,  decision  analysis,  systems  analysis,  and 
organizational  psychology. 

The  approach  to  realizing  these  objectives  has  been  to  contract  for 
specific  developmental  products  with  about  a dozen  R&D  companies,  one 
in-house  laboratory,  and  one  university.  A small  steering  committee 
consisting  of  the  ODA  Program  Manager  and  four  other  scientific  officers 
at  the  Office  of  Naval  Research,  with  non-voting  representation  from 
Op-987,  Op-9^3  and  the  Naval  Electronic  Systems  Command,  provided  overall 
direction.  The  day-to-day  monitorship  of  individual  contracts  was  handled 
in  the  traditional  ONR  way.  The  Steering  Committee  met  about  once  a month; 
less  frequently,  a larger  coordinating  committee  was  convened.  During  the 
first  three  years  of  the  program,  there  were  intensive  semi-annual  meetings 
at  which  all  contractors  gave  formal  presentations  of  their  work.  There 
have  also  been  a few  public  talks  about  the  program,  e.g.,  at  a symposium 
sponsored  by  the  Air  Force  on  command  and  control  R6D. 

The  underlying  rationale  for  all  ODA  work  has  been  the  assumption 
that  all  products  of  this  program  would  be  subjected  to  experimental  test. 
Tests  were  to  be  undertaken  at  first  by  the  various  contractors  in  their 
own  facilities,  and  later  at  a designated  test  bed  that  would  serve  the 
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entire  program.  Ultimately,  as  various  decision  aids  or  other  products 
moved  toward  fleet  use,  they  would  be  tested  in  operational  settings 
by  appropriate  Navy  agencies.*  In  summary,  it  is  important  to  emphasize 
the  experimental  basis  for  the  entire  program,  particularly  for  those 
developments  that  could  become  decision  aids  in  one  form  or  another. 


Contractor  members  of  the  ODA  program  can  be  considered  as  falling 
into  either  of  two  categories:  a)  those  providing  supporting  or  context- 

setting material  for  the  program,  b)  developers  of  specific  decision- 
aiding  displays,  procedures,  and  techniques.  The  following  pages  deal 
with  each  group  in  some  detail. 


PROGRESS  AND  PRODUCTS 


Support 

A half-dozen  separate  efforts  have  been  directed  toward  support 
activities.  That  is,  a significant  proportion  of  the  ODA  program  has 
concentrated  on  various  context-sett ing  objectives  that  would  later  be 
applied  to  actual  decision  aids  that  were  developed.  The  support  activities 
— described  in  this  section — produced: 

— a taxonomy  of  decisions  and  warfare  scenarios; 

— representative  and  realistic  data  bases; 

— a measure  of  effectiveness  for  decision  aids; 

— case  studies  of  command  and  control  systems; 

— guidelines  for  introducing  new  decision  systems  or  aids  into 
existing  organizat ions ; 

— a general  purpose  laboratory  test  facility. 

1 . Decision  taxonomy  and  warfare  scenarios 

Very  early  in  the  ODA  program  it  became  apparent  that  any  decision 
aids  that  would  be  developed  would  have  to  be  tested  in  a realistic  context 
of  pol i t ical -mi  I i tary  situations.  It  was  alsc  apparent  that  a common 
knowledge  base  dealing  with  the  nature  of  naval  tactical  decisions  would 
have  to  be  provided  as  a background  for  the  contractors  who  would  be 
developing  specific  aids.  What  are  tactical  decisions?  who  makes  them? 
what  are  the  time  constraints  under  which  decision  makers  operate?  what  is 
the  physical  environment  in  which  task  force  commanders  traditionally  work? 


* Informal  arrangements  have  been  made  for  ODA  testing  to  be  moved  to  the 
ARPA/Navy  advanced  Command  Control  Architectural  Test  facility  (ACCAT) 
in  San  Diego. 
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The  Naval  Warfare  Research  Center  of  the  Stanford  Research 
Institute  has  made  two  contributions  dealing  with  these  and  other  ques- 
tions. First,  NWRC  developed  an  analytically-based  taxonomy  of  naval 
tactical  decisions.*  Three  decision  phases  were  defined,  i.e.,  planning, 
execution  and  evaluation;  and  32  subsets  of  decision  types  were  enumerated, 
e.g.,  assigning  responsibility,  selecting  targets,  etc.  Typical  task  force 
organizations  were  described  with  particular  emphasis  on  information  flow. 
This  first  NWRC  analysis  produced,  in  addition  to  the  descriptive  material 
already  named,  some  interesting  and  serendipitous  insights.  For  example, 
task  force  commanders  rarely  work  in  real  time;  instead,  they  concentrate 
most  of  their  attention  on  advanced  planning.  This  suggests  that  decision 
aids  need  not  emphasize  instantaneous  response  rates.  Task  force  commanders 
are  deluged  with  information  in  the  form  of  messages,  so  that  improving 
or  increasing  communications  flow  is  not  indicated.  The  element  which  is 
not  handled  well  by  task  force  command  systems  is  that  of  uncertainty. 
Furthermore,  high  level  tactical  decision  makers  are  highly  idiosyncratic 
in  their  approaches — a fact  that  has  too  often  been  ignored  by  system 
designers,  who  more  often  than  not  constrain  flexibility  rather  than 
enhance  it.  A great  deal  of  attention  will  have  to  be  paid  to  the  process 
of  introducing  new  decision  aids  into  existing  naval  systems.** 

The  first  contribution  of  NWRC,  the  proposed  taxonomy,  was  followed 
by  a review  of  a large  number  of  decision  aids  and  an  assessment — analytic 
rather  than  experimental — of  their  applicability  to  the  task  force  com- 
mander's role.  Some  36  aids  were  included,  ranging  from  check  lists  to 
speech  recognition  systems,  and  an  attempt  was  made  to  assess  each  in  a 
quantitative  way.  The  intent  of  this  work  was  to  determine  priorities  by 
which  the  ONR  managers  could  order  their  developmental  program.  For 
example,  high  priority  was  urged  for  a "brief  decision  analytic  aid"  that 
would  combine  utility  theory,  value  assessment  and  probability  assessment, 
and  the  use  of  decision  trees;  another  aid  high  on  the  priority  list  was 
warfare  simulation  modeling.  Checklists  were  also  put  forth  as  a low-cost, 
high-payoff  class  of  decision  aid. 

The  second  thrust  of  the  NWRC  support  work  was  aimed  at  developing 
realistic  political-military  scenarios  with  which  the  decision  aids 
produced  in  the  program  would  have  to  cope.  Two  scenarios  were  built, 
in  considerable  realistic  detail,  in  order  to  provide  decision  contexts 
involving:  a)  strike  warfare  and,  b)  amphibious  operations.  In  the  former 

case  the  task  force  commander  is  confronted  with  a large  number  of  decisions 


* Henceforth,  the  term  "tactical  decision"  refers  to  a decision  made  at 
the  level  of  the  task  force  commander. 

**  See  Payne,  J.R.,  et  al. , The  Naval  Task  Force  Decision  Envi ronnrent , 
NWRC-TR-8,  September  197^"i  and  Payne,  J.  R.  , et  al.,  A Brief  SuTvey 
of  Potential  Decision  Aids  for  the  Task  Force  Commander  and  his 
staff,  NWRC-RM-8A,  August  197^ 
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correlated  with  the  taxonomy  mentioned  above;  i.e.,  he  faces  a number 
of  realistic  preliminary  planning  options  and  several  execution 
decisions.  The  scenarios  are  built  to  provide  some  uncertainty,  and 
there  is  sufficient  flexibility  of  outcome  to  permit  several  repeated 
runs  of  each  scenario.  The  scenarios  are  hypothetical,  of  course,  but 
their  rich  detail  has  been  drawn  from  real  events  and  real  places.* 

2.  Data  bases 

In  order  that  the  scenarios  prepared  by  SRI  could  become  the 
basis  for  demonstrating  and  testing  the  decision  aids  produced  in  the 
program,  it  was  necessary  to  provide  appropriate  information  in  the 
form  of  data  bases.  This  task  was  undertaken  by  CTEC , Inc.,  and  it 
has  been  operating  continuously.  (Da*a  bases  are  the  files  containing 
the  detailed  information,  in  a form  suitable  for  automatic  storage  and 
retrieval,  necessary  to  make  the  warfare  scenarios  realistic  and  capable 
of  execution.  For  example,  one  of  the  ONRODA**  scenarios  refers  to 
certain  enemy  aircraft;  the  associated  data  base  would  contain  performance 
characteristics  of  particular  aircraft,  along  with  information  about 
its  vulnerabilit  jei  capacity,  sortie  rate,  and  so  on.)  In  defining 

data  requireme  worked  with  other  contractors  in  order  to  under- 
stand the  den  particular  decision  aids  would  make  of  an  informa- 
tion system.  jIso  drew  on  the  characteristics  and  contents  of  a 

large  number  of  extant  naval  tactical  data  bases.***  Although  it  would 
have  been  desirable  to  produce  data  files  "containing  all  the  information 
a task  force  commander  might  wish  to  use  . . . ,"  the  files  that  were 
actually  developed  were  somewhat  constrained  by  resources,  by  the  realities 
of  the  ONRODA  scenarios,  and  by  the  desire  to  conduct  this  phase  of  the 
program  in  an  unclassified  environment.  Ultimately  some  17  major  data  files 
were  defined,  including:  "operations  area,"  "logistics,"  "weapons  charac- 

teristics," "enemy  intent,"  and  so  on.  The  files  were  of  three  general 
types:  a)  the  "initial  data"  that  a task  force  command  system  would  possess 

at  the  start  of  the  ONRODA  scenario,  b)  "incremental  data,"  which  involves 
conditions  undergoing  change  not  under  the  task  force  commander's  control, 
and,  c)  decision-dependent  data,  which  stems  from  the  actions  of  the  com- 
mander during  experimental  operations. 

* See  Payne,  J.  R. , and  Rowney,  J.  V.,  ONRODA  Warfare  Scenario,  NWRC-RM-83, 
June  1975;  Rowney.  J.  V.,  Supplement  to  ONRODA  Warfare  Scenario,  NWRC- 
RM-90,  March  1976;  and  Rowney,  J.  V.,  Amphibious  Warfare  Scenario, 
NWRL-RM-86,  October  1975- 

**  An  acronym  from  ONR  Operational  Decision  Aids. 


***  Interestingly,  several  deficiencies  turned  up,  eg.,  inconsistencies  in 
the  labels  given  to  data  elements  and  major  gaps  in  the  coverage 
accorded  some  U.S.  weapons  and  platforms. 
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CTEC  has  continued  to  define  and  coi^pile  files  of  "variable  data 
factors,"  such  as  tables  of  weapon  coverage,  daaage  assessaent  of  ships 
as  a function  of  eneay  action,  and  tables  of  engageaent  statistics.  In 
addition  to  the  aforeaentioned  files,  which  were  delivered  to  the 
University  of  Pennsylvania  in  late  1976,  new  files  are  being  built  to 
contain  data  required  by  particular  contractor  aids;  e.g.,  the  eaission 
control  planning  aid  of  the  General  Research  Corporation  (described  in  a 
later  section)  needs  detailed  inforaation  on  the  physical  characteristics 
of  U.S.  Navy  radars.  In  order  to  provide  data  that  will  perait  Pennsyl- 
vania to  exercise  its  data  base  aanageaent  systea,  CTEC  h»  built  files 
that  are  appropriate  to  the  Penn  network  approach.  Still  continuing  is 
CTEC's  work  in  supplying  sensor  coverage  data  (i.e. , radar  algorithas) 
and  other  inforaation  as  the  need  arises.  In  a sense,  the  CTEC  contribu- 
tion provides  the  coaaonality,  both  generic  and  specific,  needed  to  cnapare 
the  relative  efficiencies  of  the  aany  decision  aids  being  generated  by 
other  contractors.  It  nay  also  be  possible  to  eaploy  these  ODR  data  bases 
as  standards  by  which  other  naval  information  systens  can  be  aeasured. 

And  not  the  least  of  CTEC's  contributions  is  the  likely  disciplining  effect 
on  the  entire  progran  to  the  end  that  different  elenents,  i.e.,  contractors, 
will  confront  a single  data  base  rather  than  one  that  is  idiosyncratic  to 
their  particular  decision  systens.* 

3-  Measuring  the  effectiveness  of  decision  aids 

Fron  its  inception  the  OOA  progran  had  a strong  ooMitaent  to  the 
conduct  of  experiaents  that  would  provide  quantitative  evidence  about  the 
various  decision  aids  being  developed.  For  this  reason  a special  effort 
was  initiated  with  Analytics,  Inc.,  to  "develop  and  analyze  alternative 
methods  for  measuring  the  performance  of  existing  and  proposed  decision 
aids  . . . ."**  Recognizing  the  very  difficult  problem  of  developing  a 
measure  of  effectiveness  for  an  element  of  the  task  force  connand  center 
in  %<hich  people  and  their  equipaent  interact  in  a little  understood, 
highly  complex  way,  the  cxxitractor  sought  to  provide  a method  that  vould 
be  statistically  reliable,  objective,  and  capable  of  extrapolation  from 
laboratory  to  real-%<orld  conditions.  The  first  effort  sou^t  to  formulate 
a completely  comprehensive  basis  for  measures  of  effectiveness;  this,  it 
was  hoped,  would  suit  all  possible  aids  and  all  aspects  of  a task  force 
camaander's  decision  functions.  After  review  by  the  ODA  Steering  Coanittee 
the  approach  was  abandoned  as  too  general,  and  a new  emphasis  was  pro- 
posed. The  decision  function  ultimately  selected,  tdilch  became  the  basis 
for  the  proposed  measure  of  effectiveness  that  was  developed,  was  inforaa- 
tion storage  and  retrieval.  In  essence  the  measures  developed  were  two: 
a)  gain  in  information  retrieved  and  b)  rate  of  gain  as  compared  to  random 


* For  more  detailed  documentation  of  the  data  files  prepared  by  CTEC, 

Inc.,  see  their  OPA  Data  Base  Final  Report,  27  Feb.  1976. 


**  ONR  Procurement  Request  dated  20  November  197^. 
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processing.  Put  another  way,  the  measure  of  effectiveness  wouid  indicate 
the  relevance  of  retrieved  information  and  the  amount  of  time  taken  to 
retrieve  it;  the  measure  can  be  expressed  as  the  ratio  of  relevant-to- 
total- informat  ion  retrieved.  The  analysis  which  led  to  the  contractor's 
choice  of  this  measure  makes  it  possible,  in  the  experimental  comparison, 
to  show  that  the  performance  of  decision  aids  interacts  with  the  time 
available  to  use  them.  That  is,  a particular  aid  might  produce  more 
relevant  information  early  in  the  overall  decision  process;  but  given 
enough  time,  a competing  aid  might  outperform  the  first.  In  the  contrac- 
tor's words,"  . . . rapid  solutions  tend  to  ignore  detail  and  consequently 
have  lower  asymptotic  values  while  highly  refined  solutions,  possessing  a 
great  wealth  and  richness  of  structure,  require  more  time  to  execute."* 

The  measure  derived  by  Analytics  has  not  been  applied  experimentally — at 
first  because  none  of  the  aids  under  development  was  at  a state  that  would 
permit  valid  testing,  and  later  because  the  measure  was  seen  as  impractical 
to  apply  and  its  use  was  suspended. 


4.  Case  studies  of  operational  decision  aids 

There  is  nothing  n^w  about  the  concept  or  practice  of  enhancing 
naval  tactical  decision-making  with  automated  aids.  Fairly  recently — early 
in  1976 — the  ODA  program  managers  initiated  a contract  that  would  examine 
currently  operational  systems  employing  decision  aids  as  a means  of  better 
understanding  the  successes  and  failures  of  such  systems.  It  was  believed 
that  a careful  "lessons  learned"  approach  would  provide  useful  insights 
into  the  attributes  of  existing  aids  that  either  promoted  or  inhibited 
their  adoption  and  performance.  The  System  Planning  Corporation  (SPC) 
undertook  a nine-month  study  that  examined  these  areas:  a)  levels  of 

sophistication  of  selected  decision  aids,  b)  problems  in  adopting  decision 
aids  and  how  they  were  resolved,  c)  successful  uses  of  aids,  and  d) 
approaches  to  the  transfer  of  decision  aiding  technology  from  R&O  to  opera- 
tions. The  SPC  investigation  addressed  these  issues  in  the  context  of 
three  tactical  command  and  control  systems:  the  Navy's  Trident  defensive 

command  and  control  system,  the  Coast  Guard's  Vehicle  Traffic  System,  and 
the  Army's  Simulated  Tactical  Operations  System. 


The  Trident's  automated  display  is  a sophisticated  decision  aid 
which  permits  the  commander  to  explore  various  tactical  options.  Unlike 
some  of  the  aids  being  developed  in  the  ONR  program,  the  Trident  system 
is  limited  to  highly  structured,  repetitive  situations,  i.e.,  tracking 
anti-submarine  weapons.  The  development  of  the  Trident  display  was 
modular  and  evolutionary;  that  is,  the  display,  although  computer-generated, 
appears  to  be  like  more  conventional,  manually-produced  displays. 

The  Vehicle  Traffice  System  (VTS)  is  used  by  the  Coast  Guard  to 
monitor  ship  traffic  in  San  Francisco  Bay  and  to  provide  collision  avoidance 


* Analytics,  I nc.,  Measur i ng  the  Performance  of  Operational  Decision  Aids, 
Final  Report  1 161 -B , 30  April  1976,  p.  31^- 
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Information  to  pilots  and  masters.  The  system  is  similar  to  Trident 
in  that  the  repertory  of  possible  decision  actions  is  limited.  It 
differs  from  the  Trident  displays  in  an  important  respect:  the  new 

Coast  Guard  system  fully  supersedes  its  manual  predecessor  and  does  not 
attempt  to  be  evolutionary.  The  SPC  investigators  found  that  there  was 
an  initial  reluctance  to  accept  the  new  system  on  the  part  of  its 
operators.  At  first,  the  manual  system  was  run  in  parallel  with  VTS. 

A critical  event  in  which  the  older  version  failed  and  the  VTS  was  able 
to  fill  in,  however,  led  users  to  accept  VTS.  Several  attributes  of  the 
VTS  development  are  cited  as  significant:  a)  although  design  engineers 

felt  that  the  system  should  display  only  processed  information,  a raw 
radar  display  was  provided  as  well  as  a filtered  picture  because  opera- 
tors said  that  they  needed  access  to  all  the  surveillance  information 
being  received;  b)  the  VTS  displays  and  consoles  were  made  to  be  aestheti- 
cally appealing;  c)  a great  deal  of  attention  was  paid  to  providing  a 
system  that  was  easy  to  learn  to  use;  and,  d)  there  was  extensive  and 
continuous  interaction  between  system  designers  and  the  people  who  were 
to  eventually  operate  and  maintain  VTS. 

The  Army's  SIMTOS  is  not  an  operational  tectical  system;  it  is 
rather  an  R&D  test  facility  not  unlike  the  intended  ONR  ODA  test  bed 
being  installed  at  the  University  of  Pennsylvania.  (See  below,  p.lO.) 

The  principal  emphasis  of  SIMTOS  is  on  testing  data  access  and  retrieval 
methods  using  a computer,  interactive  terminals,  and  experienced  Army 
operations  officers  as  subjects.  The  latter  serve  as  decision  makers  in 
situations  set  up  to  simulate  the  planning  and  execution  phases  of  war- 
fare operations  in  Central  Europe.  Operations  officer  subjects  have  full 
knowledge  of  the  status  of  both  friendly  and  enemy  forces;  there  is  no 
uncertainty,  nor  is  it  necessary  for  the  system  to  filter  or  evaluate 
information.  The  latter  is  seen  as  an  unrealistic  attribute  of  SIMTOS. 

Other  continuing  problems  have  been  difficulties  in  defining  data  elements, 
problems  of  deriving  measures  of  effectiveness,  and  certain  incompatibilities 
with  existing  operational  organizations  in  the  Army.  Like  the  Coast  Guard's 
VTS,  SIMTOS  designers  set  out  to  incorporate  principles  of  ease  of  learn- 
ing and  operating  the  system;  they  also  stressed  the  aesthetics  of  SIMTOS 
displays  and  consoles. 

What  is  the  significance  of  SPC's  case  studies  for  the  ODA 
program?  One  is  that  none  of  the  decision  aids  that  were  examined  depart 
very  much  from  the  older  systems  they  have  replaced.  This  is  particularly 
true  for  the  displays  of  Trident  and  VTS.  Another  point  is  that  human 
decision  makers  in  these  systems  remained  as  central  elements  and  their 
traditional  modes  of  operating  were  not  drastically  changed.  It  is  impor- 
tant to  keep  in  mind  that  the  organizational  levels  studied  by  SPC  were 
considerably  lower  than  that  for  which  the  ONR  aids  are  intended.  Thus  it 
is  probable  that  task  force  commanders  would  be  even  more  likely  (see 
below,  p.IO  ) to  demand  that  departures  from  systems  with  which  they  are 
comfortable  and  experienced  be  relatively  minor.  The  staff  of  SPC  reports 
an  extreme  reluctance  to  automate  decision  aids  to  the  extent  that  utility 
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theory  or  decision  theory  is  involved.  In  fact,  there  seems  to  be  a 
general  reluctance  to  incorporate  automation  into  tactical  decision 
systems,  even  in  the  face  of  the  ever  tightening  time  constraints  facing 
these  systems. 

On  the  subject  of  adapting  new  aids  into  larger  operating 
systems,  the  SPC  investigation  makes  the  point  that  some  tactical  systems 
are  developed  independently  and  their  designers  fail  to  take  into  account 
interface  issues.  The  rare  cases  in  which  this  does  not  happen,  e.g.. 

Coast  Guard's  VTS,  are  character! zed  by  close  cooperation  throughout  the 
development  cycle  between  planners  and  operating  people. 

Another  important  observation,  and  one  that  flies  in  the  face  of 
an  earlier  point,  is  that  there  appears  to  be  an  overabundance  of  tacti- 
cal data  being  generated.  The  technology  of  filtering  and  processing 
is  certainly  available  and  understood;  but  the  resistance  to  using  it 
(see  above)  is  what  has  not  been  overcome.* 

5 • Introducing  new  decision  aids  and  their  impact  on  Navy  organi- 
zations 

It  is  a well  known  but  little  documented  fact  that  the  introduc- 
tion of  automated  elements  into  organizations  causes  considerable  turbu- 
lence. Sometimes  these  events  have  the  unhappy  ending  in  which  the 
intended  decision  aid  is  rejected  and  discarded  completely,  and  sometimes 
(rarely)  the  aid  is  successfully  adopted — although  never  without  a cost. 

In  an  attempt  to  understand  the  organizational  perturbations  which  occur 
when  automation  is  brought  to  command  and  control  systems,  the  ODA  program 
management  commissioned  research  by  CACI,  Inc.  The  work  had  these  main 
objectives:  a)  to  review  what  is  known,  in  a general  sense,  about  organi- 

zations and  how  automated  information  systems  impact  on  them;  b)  to  conduct 
case  studies  of  automated  naval  systems  to  learn  how  the  work  of  high  level 
commanders  and  their  staffs  has  been  affected  by  the  automation;  c)  to 
develop  hypotheses  that  could  become  the  basis  for  experimentation  with  the 
decision  aids  being  developed  by  the  other  contractors;  and  d)  to  define 
strategies  for  reducing  resistance  to  change  and  for  assuring  that  aids 

do  get  adopted.  '' 

The  CACI  investigators  worked  from  the  assumption  that  a typical 
task  force  commander  is  supported  by  a staff  of  about  twenty  officers,  the 
flexibility  of  whose  roles  depends  on  both  the  mission  to  be  performed  and 
the  idiosyncratic  style  of  the  commander  himself.  The  latter  is  an 
important  point  and  it  underlies  the  main  objective  of  this  part  of  the 
ODA  program:  to  learn  how  the  staff  organization  can  be  designed  to  fit 

a particular  commander  when  automated  aids  are  introduced. 


* See  Lucas,  G.  and  Ruff,  J.,  An  Investigation  of  Operational  Decision 
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^ CACI  dealt  with  both  formal  and  informal  aspects  of  organiza- 
tions. (The  former  are  officially  prescribed  structures:  patterns  of 

authority  and  responsibility  and  accountability;  the  latter  are  the 
relationships  that  evolve  among  people  in  organizations.)  Formal 
structure  has  to  be  able  to  accommodate  automated  decision  aids  in 
three  ways:  a)  placing  them  physically;  b)  establishing  new  roles, 

e.g.,  those  of  technicians  and  operators;  and,  c)  placing  the  decision 
aid  operators  in  the  hierarchy.  Choices  have  to  be  made  in  each  of  these 
areas;  for  example,  should  current  staff  be  retrained  as  operators  or 
should  new  specialists  be  added  to  the  staff?  The  informal  structure 
of  a staff  takes  into  account  the  degree  of  delegation  of  authority, 
degree  of  centralization  of  leadership,  and  the  nature  of  interpersonal 
communication. 

In  its  review  of  the  literature  of  organizational  behavior, 

CACI  found  both  inconclusive  and  ambiguous  statements  about  how  decision 
aids  affect  organizational  structures.  This  led  to  the  investigators' 
development  of  their  own  model  for  determining  the  most  appropriate 
organizational  structures  for  technical  (i.e.,  automated)  environments. 

The  model  says  that  the  form  of  an  organization  should  be  contingent 
upon  its  mission,  its  staff,  and  the  technology  available  to  the  staff. 

Using  the  model  as  a basis  for  designing  different  organizations,  it  is 
possible  to  hypothesize  which  are  most  likely  to  accept  technical  innova- 
tion and  which  are  not.  The  model  also  assumes  that  the  single  most 
important  factor  in  building  a naval  task  force  command  organization  is 
the  style  of  the  commander.  One  of  the  products  of  the  CACI  work  is  a 
set  of  hypotheses  that  propose  optimal  organizational  structures  for 
different  leadership  styles  and  functions.  The  hypotheses  are  carefully 
defined  so  that  they  can  be  used  in  an  interim  sense,  i.e.,  in  specifying 
experimental  conditions  for  test  purposes,  or  in  recommending  ways  to 
install  decision  aids  with  a maximum  chance  of  acceptance.  For  example, 
if  a task  force  commander  has  had  little  or  no  exposure  to  automated 
aids,  specialists  should  be  assigned  rather  than  the  commander's  staff 
trained;  if  on  the  other  hand  a commander  has  had  experience  with  computers, 
models  and  decision  analysis,  it  is  best  to  put  aids  directly  under  his 
control  and  not  to  install  an  intervening  specialist.  The  model  also 
stresses  team,  as  opposed  to  individual,  training  and  experimentation. 

The  case  studies  by  CACI  dealt  with  four  Navy  systems:  a)  the 
Naval  Tactical  Data  System,  which  has  been  used  operationally  for  15 
years;  b)  an  ad  hoc,  computer-based  system  (Outlaw-Hawk)  that  combined 
ocean  surveillance  information,  an  automated  management  information 
system,  and  NTDS  inputs;  c)  the  automated  management  Information  system 
(AMIS)  used  by  the  officer-detailing  activity  of  the  Bureau  of  Naval 
Personnel,  and  d)  a developmental  fleet  command  center  (FCC)  that  Is 
intended  for  use  in  making  strategic  decisions  in  a shore-based  operational 
control  center  during  crisis  situations. 

Coming  out  of  the  case  studies  were  a number  of  consistent  themes, 
all  of  which  have  implications  for  the  aids  being  developed  In  the  ODA 
program.  First,  a most  neglected  aspect  of  introducing  new  automated 
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decision  elements  is  the  development  of  training  and  implementation 
plans;  such  strategies  are  rarely  given  much  thought,  and  few  resources 
are  made  available  to  support  them.  Another  common  finding  was  that 
automatic  aids  created  an  environment  that  centralized  decision  making 
in  the  command  authority  himself,  although  this  is  not  necessarily  the 
most  efficient  organizational  model  for  solving  problems.  It  was  also 
found  that  each  of  the  systems  would  have  benefited  if  two  coordinators 
had  been  provided:  a technical  administrator,  and  an  "integrator"  who 

would  serve  as  interface  for  the  commander  and  the  automated  aid.  CAC I 
showed,  too,  that  there  was  an  initial  resistance  to  aids  that  appeared 
to  be  superseding  traditional  human  judgments;  such  a reaction  can  have 
negative  effects  on  morale  and  cause  performance  inefficiencies. 

The  CAC I work  has  implications  for  all  of  the  decision  aids  being 
developed  in  the  ODA  program.  First,  it  suggests  that  a great  deal  of 
attention  has  to  be  paid  to  accommodating  the  aids  to  individual  styles 
and  preferences  of  the  officers  who  are  to  use  them.  This  is  all  the 
more  important  because  of  the  high  level,  i.e.,  task  force  command,  bei'ng 
served  by  ODA.  Second,  the  experiments  that  are  to  be  done  with  the  aids 
should,  in  part,  include  teams  or  staffs  rather  than  relying  exclusively 
on  individual  "one-man,  one-machine"  designs.  Third,  attention  should  be 
given  to  careful  planning  for  the  introduction  of  the  aids.  (One  sugges- 
tion made  by  an  experienced  naval  officer  during  a presentation  of  the 
CACI  work  is  to  re-label  decision  aids  as  "analytic"  aids  so  that  there 
will  not  be  an  appearance  of  usurping  commanders'  prerogatives.)  Fourth, 
and  finally,  the  concept  of  demonstrating  new  decision  aids  should  be 
supported  as  a separate  activity  in  its  own  right  and  as  a means  of 
assuring  their  acceptance  or,  at  least,  fair  trial. 

6.  An  experimental  test  bed  facility 

From  its  inception  the  ODA  program  has  made  provisions  for  a 
central  experimental  and  demonstration  site  for  use  by  all  the  contractors. 
The  Department  of  Decision  Sciences,  University  of  Pennsylvania,  has  had 
the  responsibility  for  planning  and  building  a test  site  that  is  intended 
for  use  by  all  ODA  contractors.*  As  is  necessary  in  such  dual  purpose 
facilities,  the  test  bed  includes  both  the  necessary  equipment — displays, 
terminals,  and  communication  links  with  remote  computers — and  means  for 
conducting  experiments,  i.e.,  software  that  permits  logs  to  be  kept  for 
recording  the  actions  of  people  who  serve  as  decision  makers  in  experiments. 

The  test  bed  is  located  in  the  Department  of  Decision  Sciences 
and  it  consists  of  700  square  feet  to  be  used  as  a laboratory.  Sometimes 


* The  Pennsylvania  group  has  another  major  responsibility,  i.e.,  the 
development  of  aids,  displays,  and  data  management  techniques.  That 
work  will  be  described  in  a later  section. 


referred  to  as  a "decision  environment,"  the  facility  has  been  structured 
as  a flexible  tool  to  permit  the  easy  installation  and  adaptation  of 
decision  aids  produced  by  other  contractors  and  to  allow  locally  developed 
devices  and  software  to  be  added  and  tested.  The  facility  provides,  for 
demonstrations  and  experiments,  a "group  decision  area"  that  can  involve 
up  to  30  viewers.  This  part  of  the  laboratory  includes  large  Advent 
projection  screen  displays,  a control  console  for  the  experimenter,  and 
other  dispiay  projectors. 

An  "individual  decision  environment"  area  is  intended  for  experi- 
ments with  single  human  subjects.  There  are  two  work  stations  with 
character  displays,  associated  keyboards,  and  additional  computer  control 
units.  Both  individual  and  group  areas  are  sound-proofed  and  air-condi- 
tioned. 


The  principal  equipment  used  in  support  of  the  laboratory  is  as 
follows:  a)  a DEC  System  10  computer  and  a GP  100  terminal;  b)  a Ramtek 

display  and  a PDP-11  computer,  both  located  in  another  area  of  the 
University  and  used  to  generate  high  quality  color  graphics;  c)  a network 
system  for  linking  the  base  computers;  d)  two  Advent  Video  beam  projectors 
(for  black  and  white  or  color  displays);  e)  a color  TV  monitor;  f)  a 
character  display  terminal  (ADOS  460)  for  showing  low  resolution  material 
(e.g.,  bar  charts  and  rough  diagrams);  g)  a "mouse"  input  device  used  for 
pointing,  moving,  or  drawing;  h)  computer  network  units  (Pennsylvania's 
Gandalf  net);  i)  a video  network  to  bring  in  displays  generated  by  the 
remote  computer;  and  j)  a voice  synthesizer  as  well  as  other  more  conven- 
tional auditory  and  visual  methods  of  generating  alerting  signals. 

The  Pennsylvania  test  facility  will  also  include  provisions  for 
recording  the  actions  taken  by  decision  makers  working  with  the  various 
aids.  This  will  make  it  possible  to  compile  a history  of  how  different 
subjects  used  the  same  aid  and  how  the  aids  compared  with  one  another. 
Because  the  recorded  actions  will  be  stored  in  a computer,  it  will  be 
possible  to  program  statistical  summarizing  and  analytical  routines  so 
that  experimenters  can  have  fast  feedback  on  the  results  of  their  laboratory 
trials. 


The  Penn  test  facility  has  been  used,  and  late  in  1976  it  served 
as  the  site  for  demonstrating  a number  of  decision  aids  that  were  brought 
in  by  other  contractors. 

Decision  Aids 


The  heart  of  ONR's  ODA  program,  and  the  source  of  its  main  payoffs, 
is  the  group  of  contractors  who  are  developing  aids  of  various  kinds. 

There  have  been  nine  separate  efforts,  and  each  is  described  in  the  follow- 
ing pages.  Some  of  the  highlights  have  been: 

— a method  for  incorporating  and  displaying  risk  in  tactical 
decision  making; 
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— applications  of  decision  analysis  to  navai  tactical  problems, 
including  the  development  of  special  display  and  input  modes; 

— new  approaches  to  generating  automatic  alerters  and  warnings  from 


iarge  data  f i ies; 

— techniques  for  organizing  data  so  that  it  becomes  adaptively 
responsive  to  the  preferences  of  different  users  (i.e.,  tacticai 
commanders) ; 

— several  "outcome  calcuiators"  that  make  possible  the  very  rapid 
assessment  of  alternative  courses  of  action  facing  tactical 
decision  makers; 

— ways  to  generate  and  display  information  about  the  complex  emission 
patterns  and  interactions  of  own  force  electronic  radiation,  i,e., 
what  might  be  disclosed  to  an  enemy. 


1.  Data  base  management  and  an  alerting  technoloq 


The  Department  of  Decision  Sciences,  University  of  Pennsylvania, 
has  produced  a "Decision  Aiding  Information  System"  (DAISY)  that  is  based 
on  management  science  models.  The  main  component  of  DAISY  is  an  inter- 
active computer  with  its  associated  displays,  input  devices,  and  software. 
DAISY  is  intended  for  use  in  complex  decision  sequences,  and  it  is  organized 
so  that  the  tactical  commander  himself  can  have  direct  access  to  and  be 
alerted  by  the  computer.  DAISY  does  not  provide  fully  automatic  decisions, 
nor  is  it  intended  to  develop  in  that  direction.  Rather,  the  system  is 
seen  by  its  builders  as  a surrogate  staff  officer  who  supports  his  princi- 
pal, i.e.,  the  tactical  commander,  by  "providing  access  to  distributed  data 
bases,  powerful  mathematical  modeling  techniques,  and  a means  of  sequencing 
through  the  decision  proces  s . . . . ^ 


The  elements  of  DAISY  are:  decisions  (choices,  which  contain 

information  about  what  has  happened  earlier;  alternatives;  and  time 
constraints);  a relational  data  base;  models  (arrangements  of  rules  for 
handling  data  within  DAISY);  triggers  (indicators  of  events  or  changes  of 
state  in  the  system);  a data  base  management  system  to  facilitate  storage 
and  retrieval;  and  a user-specified  graphics  capability  which  permits  the 
multiple  segmentation  of  a display  screen. 


The  user  of  DAISY  may  call  up  CRT  displays  of  data  files,  and  from 
them  he  can  instruct  the  computer  to  extract  information  relevant  to  the 
decision  he  is  facing.  Data  management  techniques  make  it  possible  for 
large  files  to  be  organized  and  reorganized  automatically  in  response  to 
the  areas  of  interest  of  the  decision  maker  and  in  ways  to  reduce  system 
response  time.  Alerters  or  triggers  adapt  to  the  unique  queries  of 
individual  decision  makers  and  automatically  call  to  their  attention 
pending  events  that  are  critical.  The  trigger  notion  in  DAISY  is  particu- 
larly useful  and  it  represents  a new  departure  because  it  is  adaptive;  that 

* Morgan,  H.  L.,  DAISY:  An  Applications  Perspective,  University  of 

Pennsylvania  Department  of  Decision  Science,  Working  Paper  75"ll-03. 
November  1975- 
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is,  triggers  are  themselves  responsive  to  the  individual  styles  of 
decision  makers  using  DAISY. 

Several  applications  for  DAISY  are  envisioned.  For  example, 
the  system  can  be  used  in  a training  context  in  which  decision  makers 
would  face  prepared  scenarios  and  be  monitored  by  DAISY  for  post-exercise 
feedback.  A series  of  training  runs,  with  the  same  and  different 
decision-maker  subjects,  would  make  it  possible  to  understand  the  learn- 
ing process  in  tactical  situations.  Also,  training  runs  would  enable 
data  base  technicians  to  better  organize  their  files. 

DAISY  also  has  an  application  to  tactical  operations  planning. 
The  system  provides  a decision  maker  the  chance  to  propose  alternative 
courses  of  action,  both  his  own  and  those  of  an  adversary,  and  he  may 
ask  "what  if"  questions.  In  tactical  operations  DAISY  provides  a 
special  feature,  a dynamic  check  list,  which  automatically  monitors  the 
decision  maker  by  calling  to  his  attention  the  sequential  decisions  that 
have  to  be  made. 


A few  of  DAISY'S  othe 
represent  significant  advances 
One  is  that  DAISY  is  set  up  fo 
through  transferring  responsib 
possible  through  switching  and 
a common  data  base  by  several 
different  tasks.  One  user  may 
which  triggers  an  action  to  be 
staff  officer. 


r unique  features  bear  mention  because  they 
in  automated  decision  processing  systems, 
r multiple  users  who  may  have  to  interact 
i 1 i ty  for  decisions  to  one  another.  This  is 
scheduling  routines  which  give  access  to 
users  working  at  different  speeds  and  at 
, for  example,  create  an  alerting  condition 
considered  by  a different  user,  e.g. , a 


Daisy  uses  the  window  display  concept*  as  one  form  of  input  and 
output  device.  Thus,  up  to  five  or  more  areas  may  be  designated  on  a CRT 
for  displaying  different  data  sets  in  list  or  graphic  form;  a light  pen 
or  the  "mouse"  mentioned  above  can  be  used  to  designate  changes  or 
commands.  The  user  can,  in  DAISY,  move  windows  around  the  screen,  to  suit 
his  preferences.  Most  important,  each  window  provides  a display  and  input 
mode  for  separate  simultaneous  tasks  being  performed. 

DAISY  also  has  provision  for  a voice  synthesizer  output  that  can, 
via  computer-generated  simulated  voice,  signal  special  warnings  in  an 
auditory  mode.  DAISY  includes  fine  grain  color  graphic  capability.  The 
display  of  quantitative  information  can  be  called  up  in  tabular  or  bar 
graph  form,  for  example,  and  comparisons  can  be  highlighted  in  color. 

* Buneman,  0.  P.  et  a1..  Display  Facilities  for  Decision  Support;  The 
DAISY  Approach,  paper  presented  to  the  Second  Conference  on  Decision 
Support  Systems , San  Jose,  California,  January  1977. 
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Preliminary  reactions  to  DAISY'S  displays  have  been  favorable. 

One  representative  Navy  user  indicated  that  he  had  "finally  found  a 
system  which  matched  the  way  [l]  look  at  information  while  making  a 
decision  . . 

2 . An  aid  based  on  decision  theory:  displaying  time-changing  events 

One  of  the  most  advanced  efforts  in  the  ONR  ODA  Program  has  been 
research  leading  to  the  application  of  decision  theory  to  naval  tactical 
command  problems.  Decision  theory  provides  procedures  or  models  for 
structuring  complex  problems  into  component  parts-  The  models  are  used  to 
derive  decision  rules.  These  rules  set  up  thresholds  based  on  the  likeli- 
hood of  the  occurrence  of  key  events  which  affect  the  outcome  of  the 
decision;  as  thresholds  are  crossed  or  approached,  an  optimum  time  to 
shift  to  a new  course  of  action  is  signaled.  The  main  parts  of  a decision 
model  are:  a)  formal  structure  of  the  decision,  i.e.,  the  acts  from 
which  choices  are  to  be  made  and  the  events  that  affect  the  outcomes  of 
the  choices;  b)  the  value  of  each  decision  outcome,  i.e.,  tlie  decision 
maker's  own  preferences  among  the  possible  outcomes;  and  c)  probability 
judgments  about  uncertiin  events  that  will  affect  the  decision  outcomes. 
Decision  analysis  is  Lnsed  on  expert  judgments  about  preferred  outcomes 
and  about  probabilities  of  the  chance  events  that  can  affect  those  out- 
comes. Expected  values  are  then  computed  for  each  alternative  choice 
and  displayed  to  the  decision  maker.  As  events  change  in  time  the  expected 
values  can  be  changed  and  displayed  graphically  or  in  tabular  form. 

Displays  show  how  threat  changes  with  new  information  or  changed  proba- 
bilities. Typically,  decision  theory  application  has  required  the 
intervention  of  experienced  analysts  for  problem  structuring  and  to 
elicit  probability  judgments.  The  thrust  of  the  current  ONR  work  in  this 
area  is  to  have  a computer  serve  as  an  analytic  aid. 

An  ONR  contractor.  Decisions  and  Designs,  Inc.,  has  taken  the 
basic  components  of  these  rather  abstract  theoretical  developments  and 
has  built  a prototype  application  to  naval  tactical  decision  situations.** 
The  DDI  Aid  contains  three  displays.  The  first  shows  probability  changes 
over  time  as  they  relate  to  the  various  possible  external  situations  and 
to  the  alternative  action  decision  thresholds.  This  is  done  by  generating 
a track  or  vector  in  a triangular  space.  (An  equilaterial  triangle  has 
been  used  because  a three-state,  mutually  exclusive  decision  space  is 
assumed;  hence,  any  point  in  the  triangle's  area  represents  the  relative 
probability  of  each  possible  situation.)  Action  thresholds,  based  on  the 
decision  maker's  judgments,  are  used  to  divide  the  triangle's  state  areas; 
as  the  probability  track  approaches  each  threshold,  the  more  preferred  is 


* Ibid,  p.  11 

**  See : B rown , R.  V.,  et  al..  Decision  Analysis  as  an  Element  in  an  Opera- 
tional Decision  Aiding  System,  TR  7^~2,  Sept.  197^;  Brown  et  a1.. 

Decisions  Analysis  as  an  Eiement  in  an  Operational  Decision  Aiding 
System,  TR-75“13.  Nov.,  1975;  and  Peterson,  C,  R.,  et  al . , Decision 
Analysis  as  an  Element  in  an  Operational  Decision  Aiding  System,  TR-76-il, 
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that  action  choice.  Thus,  the  action  alternatives  available  to  the 
decision  maker  are  suggested  by  the  location  of  the  probability  vector, 
and  as  it  moves  he  can  prepare  to  shift  toward  the  indicated  course  of 
action. 


The  second  display  is  a likelihood  table  (or  intelligence  display) 
showing  data  in  the  system  relating  to  events  or  situations  that  can 
affect  the  situation  assessment;  this  display  contains  information  that 
drives  the  probability  vector.  The  third  display  shows  the  values 
assigned  to  various  outcomes  by  the  decision  maker. 

The  DOI  Aid  makes  possible  a variety  of  "what  if"  analyses. 

Thus,  the  decision  maker  can  change  the  values  he  assigns  to  likely 
outcomes.  Or,  he  can  change  his  estimate  of  the  impact  of  different 
types  of  information  received.  As  charges  are  made,  their  implications 
are  displayed  automat ical ly. 

The  DDI  Aid  is  particularly  attractive  because  it  makes  it 
possible  to  structure  multiple  uncertainties  that  are  typical  in  naval 
tactical  operations.  Thus,  several  displays  could  be  generated  simul- 
taneously, each  showing  in  the  same  display  mode  different  kinds  of 
information:  e.g.,  probability  vectors  for  weather  forecasts,  the 

enemy's  state  of  readiness,  and  our  force's  readiness.  The  probability 
thresholds  of  each  display  will  be  updated  as  one  or  another  level  of 
uncertainty  changes. 

Because  of  its  unconventional  approach  to  tactical  decisions 
the  DDI  Aid  has  raised  many  questions,  and  they  will  be  addressed  experi- 
mentally in  later  phases  of  the  program.  For  example:  Can  it  best  be 

used  to  aid  in  situation  assessment,  or  option  selection,  or  both  or 
neither?  We  need  to  know  how  to  program  the  displays  to  deal  with 
realistic  scenarios  like  ONRODA.  The  nature  of  an  optimal  interface  of 
the  tactical  commander  with  the  Aid  is  not  clear.  Will  he  interact 
directly  or  will  he  be  better  served  by  a specially  trained  staff  aide  who 
serves  as  a link  with  the  DDI  Aid?  There  are  a number  of  human  factors 
questions  to  be  addressed,  e.g.,  what  is  the  best  visual  mode  for  display- 
ing uncertainty?  How  can  color  serve  to  encode  information  In  this  Aid? 
How  can  cognition  be  aided  by  the  DDI  Aid;  i.e.,  what  are  the  impacts  of 
color  or  shape  or  number  on  decision  processes?  How  much  time  is  involved 
in  learning  to  use  and  interpret  the  AID? 

Pending  the  answers  to  some  of  these  questions,  the  DDI  Aid 
remains  an  exciting  but  unknown  entity  as  far  as  operational  application 
is  concerned. 


* * * 

The  aid  described  above  does  not  take  into  account  perceived 
risk  or  the  degree  of  certainty  a decision  maker  assigns  to  the  outcomes 
of  actions  he  takes.  Paralleling  DDI's  development,  a supplemental 


display  was  designed  and  tested  experimentally  at  the  Navy  Personnel 
Research  and  Development  Center  (NPRDC).  The  purpose  of  this  work  was 
to  determine  whether  the  addition  of  perceived  risk  to  the  DDI  aid 
would  affect  decision  making.  The  impetus  for  incorporating  risk  into 
the  DDI  Aid  was  the  belief  that  a tactical  decision  maker  might  want 
to  know  about  the  range  as  well  as  the  average  of  the  expected  utility  of 
decision  outcomes. 

Tactical  scenarios  were  developed,  and  officers  experienced  in 
tactical  operations  were  trained  as  subjects.  Risk  was  set  as  either 
"high,"  i.e.,  serious  consequences  to  own  force  if  the  wrong  decision 
were  made,  or  "low,"  i.e.,  relatively  minor  consequences  if  the  wrong 
decision  were  made.  A special  display,  the  NPRDC  Bar  Graph,  was 
developed  to  show  the  risks  as  well  as  utilities  and  probabilities  of 
outcomes  of  different  courses  of  action:  "By  showing  the  actual  utilities 

and  probabilities  of  outcomes  in  the  display,  it  was  possible  to  provide 
risk-related  information  explicitly  at  each  decision  opportunity  . . . ."* 

Results  of  the  NPRDC  experiment  were  equivocal  and  most  were  not 
statistically  significant.  The  incorporation  of  the  dimension  of  risk 
into  the  tactical  decision  process  had  relatively  little  effect  on  the 
decision  maker's  performance.  For  those  subjects  identified  as  "risk 
aversive,"  however,  the  DDI  Aid  did  assist  significantly  in  the  decision 
process.  That  is,  the  Aid  benefited  those  officers  who  tend  to  wish  to 
avoid  risk  but  it  hindered  those  who  prefer  risk.  The  reason  for  this  is 
not  clear.  Curiously,  one  experimental  condition  in  which  neither  aid  was 
provided  resulted  in  the  best  performance  by  the  risk-preferring  officer- 
subjects.  Officers  who  served  as  subjects  reported  that  they  liked  the 
DDI  display  and  that  they  could  learn  to  use  it  quickly;  the  NPRDC  modified 
aid,  the  bar  graph  showing  risk,  was  less  well  integrated  into  the  decision 
process. 

3 . Emissions  control  (EHCON) : decision  aiding  in  the  deployment 

sensor  systems 


Perhaps  the  most  complex  area  in  planning  tactical  fleet  air 
operations  has  to  do  with  electronic  sensor  information:  what  do  our 

emissions  tell  the  enemy  about  our  force?  what  do  we  know  about  his 
deployment?  how  do  our  (and  his)  sensor  systems  contribute  to  such 
knowledge?  The  General  Research  Corporation  (GRC)  has  been  working  on 
decision  aiding  concepts  that  will  deal  with  the  hard-to-visual ize  prob- 
lem of  how  electronic  sensors  interact  with  one  another  and  what  such 
information  contributes  to  judgments  by  each  side  about  the  other. 

* Gettys,  C.  F.  et  al..  Significance  of  Risk  in  Navy  Tactical  Decision 
Making:  An  Empirical  InvesTigation.  NPRDC,  TR-77~8,  December  197^, 

p.  9. 
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Planning  for  EMCON  is,  at  present,  a tedious  manual  task.  Staff 
officers  fill  in  matrices  with  EMCON  policy  for  each  ship  and  each  item 
of  emitting  equipment.  (Policies  may  range  from  complete  silence  to 
full  use.)  The  EMCON  planner  decides,  for  each  ship  in  the  force  and 
for  each  emitter,  what  risks  are  involved — and  what  advantages  are  gained 
— by  employing  the  policies  available  to  him.  These  trade-off  decisions 
require  a great  deal  of  information  about  a technology  that  is  frequently 
changed  or  upgraded,  and  the  burden  on  planners  is  considerable.  GRC  has 
proceeded  toward  the  development  of  an  automated  dedPsion  aid  for  EMCON  on 
the  basis  of  the  contention  that  models  can  provide  indications  of  the 
interactions  of  large  numbers  of  electronic  emitters  much  more  accurately 
than  can  planning  officers  working  with  traditional  manual  aids.  In  this 
application,  the  computer  offers  a powerful  means  for  assessing  all  that 
is  known  about  many  radars  (or  jammers)  and  how  they  interact.  The  GRC 
approach  has  been  to  consider  a range  of  levels  of ^sophi st i cat  ion  in  an 
EMCON  outcome  calculatoi — e.g.,  from  an  information  retrieval  system 
containing  the  attributes  of  both  sides'  electronic  equipment  and  how  it 
performs,  to,  at  the  other  extreme,  a highly  sophisticated  system  that 
can  generate  complete  EMCON  plans.  (GRC's  principal  investigator  sees  an 
immediate  need  for  data  retrieval  and  display  as  a more  important 
priority,  however,  than  the  provision  of  an  EMCON  outcome  calculator .) 

In  essence  the  GRC  aid*  is  a way  of  showing  what  information  the 
enemy  can  deduce  and  how  important  such  information  may  be.  The  GRC 
aid  calculates  "surveillance  quality  scores"  which  include  the  probability 
that  enemy  aircraft  will  be  detected  before  they  reach  a defended  area  as 
well  as  the  probability  of  successful  attack  from  a "least  defended  direc- 
tion." The  aid  displays  radar  coverage  contours  to  assist  the  electronic 
warfare  officer  in  understanding  the  relationship  between  emissions  and 
detection  probability.  The  aid  would  provide  a task  force  commander  the 
ability  to  "see"  radiation  patterns  and  to  exercise  greater  control  over 
them.  The  commander's  staff  would  have  to  specify  probabilistically  which 
enemy  platforms  and  types  of  electronic  equipment  are  in  use.  The  GRC  aid 
would  display  "frequency  space"  in  an  interactive  mode;  i.e.,  the  commander 
or  his  staff  could  define  various  conditions  of  electronic  silence  and  have 
displayed  the  information  that  might  be  denied  to  them.  Two  main  components 
of  the  GRC  aid  are  an  "emission  analysis"  model  and  an  "air  defense  sur- 
veillance" model.  The  former  assesses  the  information  given  away  by  our 
force  emissions  and  the  latter  determines  the  price  paid,  in  terms  of 
reduced  air  defense,  for  EMCON  restrictions. 

Remaining  to  be  done,  of  course,  is  the  further  development  of 
appropriate  software  and  display  hardware  that  can  make  it  possible  to 
generate  the  GRC  aid  so  that  it  can  be  tested  with  representative  naval 

* What  is  referred  to  here  as  an  "aid"  has  not  get  been  built  or 
demonstrated  except  as  a simulation. 
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officers.  The  prospects  for  doing  this  are  good  and  the  payoffs  for 
task  force  tactical  command  and  control  in  a difficult  area  are  likely 
to  be  very  high. 


Decision-making  under  conditions  of  uncertainty 
aid  for  computers 


human 


Working  on  rules  for  allocating  decision  functions  among  system 
elements,  the  Integrated  Sciences  Corporation  (ISC)  has  conducted  experi- 
ments to  determine  the  best  interface  arrangements  for  men  and  computers. 
The  thrust  of  this  effort  is  to  determine  how  best  to  provide  for  the 
incorporation  of  experienced  human  judgment  into  computer-based  or  algo- 
rithmic solutions  to  tactical  engagement  problems.  Subjects  have  been 
trained  to  "draw"  continuous  subjective  functions  on  a rather  sophistica- 
ted display  surface  as  a means  of  entering  their  estimates  of  certain 
aspects  of  a tactical  situation  into  the  computer.  (A  pilot  experiment 
established  the  relative  superiority  of  a track  ball  input  device  over  a 
light  pen  and  a joy  stick.*)  Specifically,  in  an  air  strike  planning 
task,  surrogate  officers  were  required  to  estimate  an  optimal  flight  path 
that  would  minimize  the  probability  of  detection  by  the  enemy's  sensors 
and  at  the  same  time  maximize  speed  to  the  target.  As  is  usually  the 
case  in  planning  for  actual  air  operations,  the  subjects  faced  uncertain- 
ties about  the  enemy's  sensors.  It  was  shown  that  the  subjects'  ability 
to  design  optimum  flight  paths  exceeded  that  of  a fully-automat i c mode 
and  that  men's  subjective  assessments  would  be  a good  way  to  derive 
tactical  models  for  computers. 

Direct  application  cannot  yet  be  made  to  the  design  of  a task 
force  command  and  control  system,  however,  and  experimentation  is 
continuing. 

5.  An  "Options  Matrix"  approach  to  decision  aiding 

A critical  question  in  providing  automated  decision  aids  to  a 
tactical  commander  is  the  extent  to  which  he  is  willing  to  accept  the 
recommendations  put  forth  for  his  consideration.  Another  question  is 
whether  automated  aids  can  provide  too  much  information  for  a commander 
to  assimilate.  The  Grumman  Aerospace  Corporation  conducted  research 
that  cast  some  light  on  these  matters. 

A three  condition  experiment  looked  into  ways  of  generating 
information  which  would  enable  a tactical  commander  to  choose  among 
several  alternatives  in  planning  a mission.**  One  mode  provided  the 

*lrving,  G.  W. , et  al.,  PDA  Pilot  Study:  Selection  of  an  Interactive 
Graphics  Control  Device  for  Continuous  Subjective  Functions  Applica- 
tions, TR  215-2,  April  1976. 

**See:  Kalenty,  C.  R. , and  Lockwood,  W.  L.  , Experimental  Evaluation 

of  an  Options  Matrix  as  an  Operational  Decision  Aid,  TR-CSS76-31 , March 
1976;  Kalenty,  C.  R. , et  al..  Experimental  Validation  of  an  Options 
Selection  Matrix  and  Investigation  of  Other  Display  Formats  as 
Operational  Decision  Aids,  TR-CSS77~I,  February  1977- 


( 

) 

! 


- 19  - 


tabular  and  graphic  form  but 
designate  one  ship  for  a 


task  force  commander  with  information  in 
otherwise  unprocessed.  (His  task  was  to 
special  mission  in  a situation  in  which  many  complex  attributes  of  his 
own  force,  the  enemy,  etc.,  had  to  be  weighed.)  Subjects  worked  in  a 
"command  support  simuiator"  which  generated  large-scale  displays  of  data, 
geographical  distribution  of  forces,  equipment  status  files,  and  the  like. 
The  subjects  could  request  tabular  or  graphic  data  via  a console  operator- 
aid. 


The  second  condition  included  computer-generated  checklists 
designed  to  call  to  the  commander's  attention  critical  factors  that  would 
affect  his  decision.  A selection  matrix  was  also  part  of  this  aid,  and 
subjects  had  to  enter  weightings  or  figures  of  merit  for  the  various 
factors  in  making  their  decisions.  The  computer  then  calculated  a "best 
solution"  based  on  the  subject's  estimates  and  factor  weightings. 

The  third  condition  presented  subjects  with  filled-in  matrices, 
i.e.,  all  decision  options  worked  out  automatically;  and  subjects  had  to 
decide  whether  or  not  to  accept  these  solutions. 


The  experimental  outcomes  showed  several  things.  First,  when 
presented  only  with  formatted  data  but  no  automated  aids,  subjects 
arrived  at  dec i s ions  rapidly  but  were  relatively  inaccurate.  There 
were  complaints  of  "too  much  information"  to  be  absorbed,  and  subjects 
felt  their  decisions  had  to  be  based  on  partial  information.  Second, 
the  options  matrix  that  provided  automatic  handling  and  combining  of 
subjects'  weightings  enabled  the  commanders  to  focus  on  data  pertinent 
to  the  decisions  they  were  facing.  The  fully  automated  arrangement, 
however,  did  not  meet  with  approval,  mainly  because  subjects  felt  they 
did  not  have  control  over  inputs  to  the  computer  in  formulating  decisions. 
Finally,  the  experiments  underscored  the  reluctance  of  tactical  commanders 
to  accept  computer-generated  aids  without  knowing  something  about  their 
credibility:  who  were  the  systems  analysts  who  generated  the  basic 

equations?  the  operators  who  supplied  input  data  to  the  algorithms? 
and  the  staff  who  designed  the  resulting  options  matrix?  Paraphrasing 
the  words  of  the  Grumman  investigators,  good  tactical  decision  aids 
should  enable  the  user  to  formulate  his  own  decisions  and,  in  doing  so, 
to  control  the  amount  of  automatic  aiding  he  wishes.  "The  data  displays 
can,  however,  greatly  reduce  the  effort  that  would  normally  be  required 
. . . for  data  retrieval,  organization,  condensation,  and  assimilation 
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6 . An  outcome  calculator  to  augment  tactical  decision  making 


A very  sizable  amount  of  data  goes  into 
operation;  inevitably,  as  the  battle  progresses, 


planning  a tactical 
classes  and  subsets 


* Kalenty  et  al . , op.  cit.,  1977,  p.  6. 


of  data  interact  with  one  another.  The  Naval  Warfare  Research  Center 
(NWRC)  of  the  Stanford  Research  Institute  has  conducted  analytic  studies 
leading  to  the  development  of  an  automated  outcome  calculator  that  may 
prove  useful  as  a type  of  aid  to  the  naval  task  force  commander.  The 
NWRC  mode)  is  based  on  an  air  strike  warfare  mission  that  was  drawn 
from  the  scenarios  described  earlier  (see  page  3) • 

The  outcome  calculator  consists  of  many  tables,  all  of  them 
generated  automatically  and  rapidly,  that  are  designed  to  take  account 
of  these  variables:  the  timing  of  events,  the  positioning  of  own  and 

enemy  forces,  attrition  (both  sides),  weather,  and  force  composition 
and  capability  (own  and  enemy).  Among  the  tables  generated  and  updated 
by  the  outcome  calculator  are  these:  seven  "background"  tables,  includ- 

ing weapon  platform  availability  and  damage  repair  capabilities;  three 
"scenario"  tables,  including  own  and  enemy  force  descriptions;  four 
"course  of  action"  tables,  including  operations  plans;  and  four  "results" 
tables,  including  own  and  enemy  missions  accomplished.  The  task  force 
commander  must  also  furnish  various  input  data,  e.g.,  the  optional 
aircraft  mixes,  that  he  wishes  to  consider. 

The  NWRC  outcome  calculator  generates  projections  of  losses  for 
both  sides,  taking  into  account  the  factors  named  above,  e.g.,  timing 
and  weather. 

To  date,  the  outcome  calculator  has  been  run  as  a simulation 
(but  without  real  decision  makers  involved).  The  model  is  said  to 
provide  useful  information  on  alternative  courses  of  action  in  that  its 
algorithms  permit  "what  if"  questions  to  be  addressed  for  many  conditions. 
The  extent  to  which  the  outcome  calculator  is  ultimately  of  value  will 
depend  on  the  demonstrated  ability  of  task  force  command  staffs  to  under- 
stand and  exercise  it.  NWRC's  algorithm  has  been  shown  to  be  compatible 
with  the  University  of  Pennsylvania's  test  bed  facility,  and  the  next 
step  will  be  to  include  it  in  experimental  trials  with  decision  makers. 

7 • An  automated  process  for  identifying  factors  critical  to 

tactical  decisions 


"Problem  structuring"  is  the  process  that  identifies  and  organizes 
relevant  factors  as  a way  of  helping  a decision  maker.  The  structuring 
process  can  facilitate  decisions  in  complex,  changing,  and  uncertain 
situations  of  the  type  likely  to  be  faced  by  naval  task  force  conmanders. 

It  also  works  to  counter  irrational  decisions  and  it  keeps  long  range 
issues  in  focus  where  they  might  be  cast  aside  in  favor  of  near-term 
factors.  Decision  structuring  can  be  used  in  one-of-a-kind  decisions 
for  which  there  are  no  established  procedures  or  precedents.  It  provides 
a "current  best  estimated"  course  of  action,  and  the  estimate  improves 
as  structuring  goes  forward. 

In  an  imaginative  attempt  to  automate  the  structuring  process, 
the  Decision  Analysis  Group,  Stanford  Research  Institute,  has  been 
developing  an  algorithm  that  will  enable  the  task  force  commander  to 
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build  formal  decision  models.  The  main  output  of  this  work  has  been 
a method  that  identifies  important  decision  factors.  (There  is  an 
obvious  complementarity  to  the  work  on  triggers  or  alerters  at  the 
University  of  Pennsylvania,  and  both  groups  of  investigators  have 
discussed  a combined  effort.) 

As  currently  developed,  the  SRI  structuring  process  is  based 
on  approximations  derived  from  experimental  trials.  The  SRI  process 
combines  human  and  automated  elements  in  a dialogue  format  that  runs  as 
follows:  trade-off  judgments  about  the  would-be  tactical  operation  are 

stored  in  the  computer;  these  may  be  pre-planned  or  they  can  be  developed 
by  the  task  force  commander;  the  computer  generates  questions  which  the 
analyst  interprets  to  the  commander  who,  in  turn,  provides  answers  that 
are  entered  into  the  computer  via  the  analyst;  the  dialogue  proceeds  as 
the  computer  performs  calculations  and  generates  new  questions.  As  this 
iterative  process  takes  place,  a skeletal  decision  tree  expands  with 
more  and  more  detail:  outcome  probabilities  are  assigned  to  chance  nodes, 

and  values  are  generated  for  terminal  points.  Thus,  expected  values  for 
alternative  courses  are  calculated  and  refined  as  more  detail  becomes 
aval lable. 


In  the  overall  structuring  process,  the  staff  analyst  identifies 
alternatives,  he  estimates  outcomes,  and  he  provides  ranges  of  uncertainty 
for  each  alternative  outcome.  The  commander's  priorities  or  trade-off 
preferences  take  the  form  of  a value  model.  The  computer  calculates 
expected  values  or  utilities  for  each  alternative;  it  also  points  out 
areas  needing  clarification  by  calculating  the  value  of  resolving  the 
uncertainties  in  the  overall  problem;  and  the  computer  generates  new 
questions  that  identify  events  not  yet  in  the  model.  The  staff  analyst 
and  the  task  force  commander  can  postulate  additional  events  for  the  model, 
and  the  computer  can  estimate  whether  these  events  will  influence  the 
decision  under  consideration.  If  the  event  is  added  to  the  model  on  the  basis 
of  the  analyst's  assessment,  the  process  continues. 

There  are  a number  of  unanswered  questions  regarding  SRI's 
structuring  process.  The  contractor  plans  to  implement  a computer  model 
of  his  procedure  and  to  conduct  experiments  that  will  point  out  areas 
needing  re-work  or  strengthening.  Actual  experimentation  is  to  be 
carried  out  at  the  University  of  Pennsylvania  test  facility.  Three 
subsystems,  none  of  them  yet  completed  (as  of  April  1977),  are  to  be 
built  and  tested.  They  are:  a)  a user- interface  subsystem;  it  provides, 

prompts,  and  checks  on  the  appropriateness  of  user  responses;  b)  a 
tree-handling  subsystem,  which  generates  appropriate  decision  trees, 
including  decision  probability,  and  terminal  nodes;  c)  a probability 
distribution  fitting  subsystem,  which  converts  estimated  outcomes  and 
ranges  provided  by  the  decision  maker  into  continuous  probability 
distributions. 


22 


8 . Notnography  applied  to  tactical  decision  aiding 

One  of  the  more  recent  efforts  of  the  ONR  program  has  been  the 
investigation  of  nomography  for  displaying  complex  relationships  among 
different  classes  of  data.  This  work  is  looking  into  ways  to  show, 
simultaneously,  many  variables  that  impact  on  tactical  warfare  planning, 
e.g.,  own  force  readiness,  ratios  of  f r iend 1 y-to-enemy  forces,  weather 
conditions  en  route  and  over  the  target,  desired  combat  exchange  ratios, 
and  so  on.  The  contractor.  Analytics,  Inc.,  believes  that  nomography 
will  enhance  the  selection  of  optimal  timing  for  tactics  which  have 
already  been  chosen,  such  as  when  to  launch  an  air  strike.  Another 
claim  that  has  been  made  for  this  approach  is  that  it  can  deal  with 
dynamic  data  conditions,  i.e.,  hourly  or  more  frequent  changes  of  infor- 
mation. Uncertainty  is  also  said  to  be  taken  into  account  and  its  impact 
shown . 


At  this  time  it  is  not  possible  to  assess  Analytics'  nomographic 
work.  The  approach  is  promising;  but,  pending  further  development  and 
testing  with  experienced  officers,  little  more  can  be  said. 


SOME  CONCLUDING  THOUGHTS 


There  is  no  doubt  that  the  program  has  been  rich  and  varied  and 
productive.  As  in  other  developmental  efforts,  a great  deal  of  hope  was 
generated  that  near-term  products  would  be  forthcoming  and  could  be 
incorporated  into  naval  command  and  control  systems,  both  those  in  being 
and  those  coming  on  line.  There  is  some  evidence  that  the  Naval  Electronic 
Systems  Command  would  like  to  incorporate  some  of  the  ODA  output,  e.g., 
certain  features  of  the  University  of  Pennsylvania's  DAISY,  into  its 
Tactical  Flag  Command  Center  now  in  development.  But  such  developments 
are  slow  and  they  tend  to  be  delayed  and  sidetracked  long  the  way. 
Nevertheless,  it  is  encouraging  that  some  of  the  ONF  rogram  has  met 
with  acceptance. 

A number  of  the  ODA  "products"  offer  further  ; omise.  For  example, 
the  GRC  approach  to  emission  control  offers  a straighforward  way  of 
dealing  with  the  complexities  of  electronic  radiation.  The  DDI  applica- 
tion of  decision  theory  is  perhaps  the  least  familiar  of  the  ODA  products 
to  operational  Navy  personnel.  As  such,  it  faces  the  most  demanding  and 
skeptical  questions  of  would-be  users.  This  situation  can,  of  course, 
be  eased  with  experimentally  derived  evidence  of  the  Aid's  effectiveness. 

Many  questions  remain  unanswered  in  the  domain  of  decision  aiding. 

The  major  issue,  and  one  that  has  been  raised  repeatedly,  is  what  will 
happen  when  representative  naval  officers  use  the  aids  under  somewhat 
(simulated)  realistic  conditions.  There  is  evidence  that  some  of  the 
aids  present  displays  and  require  procedures  that  depart  from  conventional 
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tactical  command  and  control  to  the  extent  that  users  require  a great 
deal  of  indoctrination.  The  NPRDC  Bar  Graph  is  an  example  of  the  latter, 
as  is  the  DDI  display.  Another  somewhat  disturbing  finding  was  that 
senior  officer  subjects  exhibited  considerable  anxiety  about  accepting 
automatically  generated  decisions  or  parts  of  decisions.  It  is  not 
enough,  in  these  cases,  to  propose  training  as  a way  out.  What  must  be 
done  is  to  find  ways  of  displaying  probabilistic  situation  assessments, 
and  probability  distributions  of  outcomes,  that  will  permit  the  decision 
makers  to  conduct  their  own  tradeoffs  to  reach  a decision. 

One  of  the  least  fulfilling  aspects  of  this  program  has  been  its 
inability  to  define  ways  to  measure  the  effectiveness  of  decision  aids. 

This  is  part  of  the  more  general  and  very  long-standing  problem  of 
criteria  for  command  and  control.  The  problem  will  not  be  solved  by  the 
ODA  program,  but  the  program  should  make  some  useful  contributions  to  the 
development  of  performance  criteria  for  the  decision  aiding  components  of 
command  and  control  systems.  This  paper  has  alluded  to  some  possible 
approaches  to  the  measurement  of  effectiveness,  but  much  more  needs  to 
be  done.  For  example,  the  question  of  the  quality  of  tactical  decisions 
arises;  i.e.,  do  the  aids  significantly  enhance  a tactical  commander's 
performance?  Are  economies  of  time  or  resource  allocation  realized 
with  one  or  another  of  the  aids?  What  are  the  basic  economics  of  intro- 
ducing decision  aids  into  the  fleet?  are  there  likely  to  be  significant 
software  costs?  (Probably.)  Are  the  human  resources — systems  analysts, 
programmers,  and  decision  theorists — available  to  support  the  aids? 

Overall  there  is  a strong  need  to  set  up  the  decision  aids — or  at 
least  the  most  promising  among  them — in  a demonstration  mode.  A great 
deal  of  attention  should  be  paid  to  hardware  reliability  and  to  providing 
understandable  demonstrations  of  what  the  aids  can  do.  This  should  not 
substitute  for  systematic,  careful  experimentation,  but  it  would  serve 
the  important  need  of  providing  familiarization  with  the  aids  among  a 
large  community  of  potential  users.  (ONR  staff  and  its  contractors  may 
have  been  too  close  to  ODA  developments  to  realize  just  how  difficult 
the  techniques  are  for  the  uninitiated  to  comprehend.)  In  the  end,  of 
course,  the  real  payoffs  for  ONR's  contribution  will  lie  in  experimentally 
derived  evidence  that  the  decision  aids  work.  This  is  difficult  and 
expensive  experimentation,  but  it  must  be  done. 

There  are  a number  of  unanswered  questions  that  are  suggested  by 
the  work  done  in  the  ODA  program  so  far.  None  of  these  issues,  of  course, 
should  have  been  addressed  earlier,  nor  could  they  have  been  without  the 
developmental  work  that  this  paper  covers.  For  example,  there  will 
inevitably  be  new  costs  to  the  Navy  if  any  of  the  decision  aids  progresses 
to  the  point  of  becoming  operational.  There  may  be  new  hardware — displays, 
data  input  devices,  computers,  and  the  links  necessary  to  incorporate 
all  of  these  into  existing  or  new  command  control  systems.  These  costs 
are  bound  to  be  non-trivial,  particularly  where  they  involve  new  techno- 
logical demands  such  as  color  displays  (an  attribute  of  some  of  the  aids 
in  the  ODA  program) . 
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But  at  least  two  other  kinds  of  costs  Joom  as  even  more  expensive 
than  hardware.  They  are,  first,  the  resources  necessary  to  build  and 
maintain  the  software  packages  that  support  the  decision  aids.  We  do 
not  have  any  estimates  of  the  extent  of  software  costs,  nor  were  they 
included  in  this  inquiry.  But  fifteen  years  of  experience  with  the 
Navy  Tactical  Data  System  has  shown  that  extensive  software  back-up  is 
requi red. 

The  second  cost,  and  one  that  is  difficult  to  predict  but  is  never- 
theless an  important  aspect  of  adopting  decision  aids  for  fleet  use,  is 
the  time  necessary  for  introducing  new  technology  of  this  type.  Assuming 
that  a decision  aid  is  tested  experimentally  and  that  the  Navy's  system 
development  process  marks  it  for  introduction  to  the  operating  forces, 
there  will  inevitably  be  long  delays  before  the  aid  can  be  expected  to 
be  in  place  and  available  to  task  force  commanders.  The  delays  due  to 
hardware  and  software  procurement  can,  perhaps,  be  minimized  if  enough 
priority  is  assigned  to  the  production  effort. 

A delay  not  so  easily  dealt  with  will  be  that  which  comes  about 
from  introducing  such  new  approaches  to  tactical  deci sion-mak. ..g  as 
are  embodied  in  some  of  the  ODA  aids.  What  we  are  referring  to  are 
the  inevitable  perturbations  to  organizations  when  some  traditionally 
human  functions  are  automated.  There  is  a growing  literature  in  the 
management  and  behavioral  sciences  dealing  with  these  effects  and  ways 
to  deal  with  them.  As  indicated  in  earlier  sections  of  this  paper,  two 
contractors — CACI  and  Systems  Planning  Corporation — have  been  concerned 
with  change  brought  about  by  automation.  We  believe  that  these  efforts 
bear  serious  study  and  should  perhaps  be  extended.  It  is  our  strong 
conviction  that  the  main  barriers  to  satisfactorily  adopting  decision 
aids  will  be  human  organizational  issues  rather  than  hardware  problems. 

Have  the  ODA  program  objectives  been  realized?  As  stated  in  the 
authorization  document  that  approved  the  ODA  program,  "the  essential 
objective  of  the  work  is  to  improve  tactical  decision  making  by  blending 
a number  of  technologies  . . . into  a practical  system  for  shipboard  use 
. . . The  ONR  program  has  brought  to  bear  several  technical  areas — 

decision  analysis,  computer  graphics,  data  management,  and  organizational 
psychology — into  a single  effort.  It  is  too  soon  to  address  the  issue  of 
whether  this  program  will  produce  techniques  that  could  be  adopted  or 
incorporated  into  other  developmental  programs;  but,  as  we  have  indicated 
earlier,  there  has  been  high  interest  expressed  toward  some  aspects  of 
the  program  by  the  Naval  Electronic  Systems  Command.  The  significance 
of  that  interest  will,  of  course,  be  demonstrated  to  the  extent  that 
ODA  concepts  and  aids  actually  find  their  way  into  the  fleet. 

* Navy  Decision  Coordinating  Paper  RPNO8,  Operational  Decision  Aids, 

8 July  1976. 
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Another  objective  of  the  program  has  been  to  develop  a better 
understanding  of  human  decision  processes  and  . . to  integrate 
(that  knowledge)  with  the  enhanced  capabilities  of  new  communications 
and  data  processing  hardware  . . . In  our  view,  this  objective  is 

the  most  ambitious  part  of  the  entire  ODA  undertaking  and,  if  successful, 
will  far  outweigh  any  hardware  contributions  the  program  may  bring  about. 
There  is  still  a long  way  to  go  to  demonstrate  that  the  efficiency  of 
command  decision-making  processes  has  been  improved;  this  program,  however, 
particularly  with  its  emphasis  on  the  decision  sciences,  has  undertaken 
a risky  venture  which  will  have  high  value  if  it  succeeds.  Perhaps 
"success"  is  a naive  concept.  All  research  generates  new  knowledge  in 
painfully  small  and  costly  Increments.  The  ODA  work  has  taken  a fresh 
look  at  the  state  of  decision  technology,  and  some  of  its  outcomes  are 
innovative  and  exciting.  What  remains,  and  what  will  undoubtedly  be  the 
next  phase  of  this  work,  are  the  demonstration  and  experimental  studies 
that  have  to  be  accomplished  before  one  can  say  much  more  about  program 
objectives  and  the  extent  to  which  they  have  been  attained. 
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